Table 2 Measure of Illiquidity¶

This notebook walks through illiquidity calculations based on methodology in The Illiquidity of Corporate Bonds, Bao, Pan, and Wang (2010). In the paper, calculations are based on corporate bond data from 2003-04-14 to 2009-06-30.

  • In order to avoid re-running the notebook every time it changes (it changes often, even by the act of opening it) and to only rerun it if meaningful changes have been made, the build system only looks for changes in the plaintext version of the notebook. That is, the notebook is converted to a Python script via nbconvert, which is often packaged with Jupyter.

Then, DoIt looks for changes to the Python version. If it detects a difference, then the notebook is re-run. (Note, that you could also convert to a Markdown file with JupyText. However, this package is often not packaged with Jupyter.)

  • Since we want to use Jupyter Notebooks for exploratory reports, we want to keep fully-computed versions of the notebook (with the output intact). However, earlier I said that I strip the notebook of its output before committing to version control. Well, to keep the output, every time PyDoit runs the notebook, it outputs an HTML version of the freshly run notebook and saves that HTML report in the output directory. That way, you will be able to view the finished report at any time without having to open Jupyter.

Overview of Outputs

* Table 2 Measure of Illiquidity:¶

  • Panel A Individual Bonds (The mean and average monthly illiquidity per bond per year)¶
    • Using trade-by-trade data
    • Using daily data
  • Panel B Bond Portfolio¶
    • Equal-weighted: Consider a daily portfolio composed of all bonds, with equally weighted bond returns used to calculate monthly illiquidity and median illiquidity per year
    • Issuance-weighted: Consider a daily portfolio composed of all bonds, with issuance weighted bond returns used to calculate monthly illiquidity and median illiquidity per year
  • Panel C Implied by quoted bid-ask spread¶
    • Mean and median monthly bond bid-ask spread per year

* Summary Statistics of Monthly Per Bond Illiquidity Using Daily Data¶

* Panel A and Summary Statistics Using MMN corrected data¶

* Replicate the Tables in the Paper (2003-04-14 to 2009-06-30)¶

* Update the Tables to the present (2003-04-14 to present)¶

¶

In [1]:
from IPython.display import Image
Image("../assets/table2_screenshot.jpg")
Out[1]:
No description has been provided for this image
In [2]:
import config

OUTPUT_DIR = config.OUTPUT_DIR
DATA_DIR = config.DATA_DIR
In [3]:
import pandas as pd
from tqdm import tqdm
import numpy as np
import matplotlib.pyplot as plt
import plotly.graph_objects as go
from plotly.subplots import make_subplots
import plotly.express as px
import seaborn as sns
from datetime import datetime
from scipy import stats
from pandas.tseries.holiday import USFederalHolidayCalendar
from pandas.tseries.offsets import CustomBusinessDay
from statsmodels.stats.stattools import durbin_watson
from statsmodels.regression.linear_model import OLS
from statsmodels.stats.sandwich_covariance import cov_hac
from statsmodels.tools.tools import add_constant
import config

import warnings
warnings.filterwarnings('ignore')

C:\Users\zhang\anaconda3\Lib\site-packages\pandas\core\arrays\masked.py:60: UserWarning: Pandas requires version '1.3.6' or newer of 'bottleneck' (version '1.3.5' currently installed).
  from pandas.core import (
In [4]:
import load_wrds_bondret
import load_opensource
import data_processing as data
import table2_calc_illiquidity as calc_illiquidity
import table2_plot_illiquidity as plot
In [5]:
# Define time frames used in the paper and the updated time stamp
today = datetime.today().strftime('%Y-%m-%d')
start_date = '2003-04-14'
end_date = '2009-06-30'

Step 1: Clean Merged Data for Daily Illiquidity Calculation¶

Before calculating illiquidity measures, it's essential to ensure that our corporate bond data is accurate and relevant. The clean_merged_data function takes care of preparing the pre-cleaned merged monthly and daily data by performing several critical cleaning steps:

  • Loads and merges the relevant datasets within the specified date range.
  • Removes any records with missing crucial price information and sorts the data chronologically.
  • Adjusts for trade execution dates by incorporating a time lag to identify consecutive trades for the same bond, and filters out those that do not fall within a one-week window, accounting for holidays.
  • Consolidates the cleaned data, readying it for the subsequent illiquidity analysis.

This step is crucial to ensure that the subsequent calculations are based on a dataset that reflects true trading activity without distortions from missing data or trades too far apart in time.

In [6]:
cleaned_df_paper = calc_illiquidity.clean_merged_data(start_date, end_date)
cleaned_df_paper.head()
Out[6]:
cusip trd_exctn_dt prclean month_time date price_eom tmt t_volume t_dvolume t_spread ... coupon ncoups amount_outstanding r_mr n_mr offering_date year month_year trd_exctn_dt_lag n
602 36962GUR3 2003-04-15 105.974001 2003-04 2003-04-30 105.861857 1.025000 62562000.0 6.632446e+07 0.004697 ... 7.250 2.0 700000.0 AAA 1.0 2000-04-26 2003 2003-04 2003-04-14 1.0
603 370442AR6 2003-04-15 95.332902 2003-04 2003-04-30 99.854148 22.666667 24255000.0 2.298073e+07 0.031849 ... 7.400 2.0 500000.0 A3 7.0 1995-09-06 2003 2003-04 2003-04-14 1.0
604 92344SAE0 2003-04-15 106.907199 2003-04 2003-04-30 107.646952 3.680556 260156000.0 2.795703e+08 0.002230 ... 5.375 2.0 2446000.0 A3 7.0 2002-11-13 2003 2003-04 2003-04-14 1.0
605 949746CC3 2003-04-15 111.335999 2003-04 2003-04-30 111.939262 2.352778 342152000.0 3.817470e+08 0.004871 ... 7.250 2.0 1000000.0 AA2 3.0 2000-08-17 2003 2003-04 2003-04-14 1.0
606 46625HAV2 2003-04-15 101.786597 2003-04 2003-04-30 102.816871 4.827778 354987000.0 3.627642e+08 0.003018 ... 4.000 2.0 1000000.0 A1 5.0 2003-01-23 2003 2003-04 2003-04-14 1.0

5 rows × 24 columns

In [7]:
cleaned_df_new = calc_illiquidity.clean_merged_data(start_date, today)
cleaned_df_new.head()
Out[7]:
cusip trd_exctn_dt prclean month_time date price_eom tmt t_volume t_dvolume t_spread ... coupon ncoups amount_outstanding r_mr n_mr offering_date year month_year trd_exctn_dt_lag n
582 285659AD0 2003-04-15 104.698901 2003-04 2003-04-30 105.318649 1.483333 93166000.0 9.734392e+07 0.006323 ... 6.850 2.0 500000.0 BAA2 9.0 1999-10-06 2003 2003-04 2003-04-14 1.0
584 36962GYZ1 2003-04-15 107.309301 2003-04 2003-04-30 107.370670 4.186111 309885000.0 3.326840e+08 0.002825 ... 5.000 2.0 2250000.0 AAA 1.0 2002-05-31 2003 2003-04 2003-04-14 1.0
586 073902AZ1 2003-04-15 112.397798 2003-04 2003-04-30 112.414809 3.891667 3432000.0 3.866618e+06 0.009621 ... 7.000 2.0 350000.0 A2 6.0 1997-02-24 2003 2003-04 2003-04-14 1.0
587 079867AH0 2003-04-15 109.513498 2003-04 2003-04-30 110.117600 5.797222 67949000.0 7.508912e+07 0.009338 ... 5.875 2.0 350000.0 AA3 4.0 1993-10-08 2003 2003-04 2003-04-14 1.0
588 345397ST1 2003-04-15 98.635997 2003-04 2003-04-30 103.956814 7.230556 653808000.0 6.506966e+08 0.005885 ... 7.875 2.0 2500000.0 A3 7.0 2000-06-07 2003 2003-04 2003-04-14 1.0

5 rows × 24 columns

Step 2: Calculate Price Changes and Perform Additional Cleaning¶

In this part of the analysis pipeline, we use the calc_deltaprc function to compute daily price changes for corporate bonds, designed to operate on cleaned and merged daily corporate bond trade data.

This calculation is based on the Measure of Illiquidity on page 10 and 11 of the peper: $ \gamma = -\text{Cov}(p_t - p_{t-1}, p_{t+1} - p_t) $. The process involves several steps:

  • Calculation of Log Prices: Transform cleaned prices to log prices for more stable numerical properties.
  • Lagged and Lead Price Changes: Determine the price changes by computing lagged and lead log prices.
  • Restricting Returns: Ensure that calculated price changes (returns) are within the range of -100% to 100%.
  • Conversion to Percentage: Change the representation of price changes from decimal to percentage for clarity.
  • Cleaning Data: Remove entries with incomplete information to maintain the quality of the dataset.
  • Filtering by Trade Count: Exclude bonds with fewer than 10 trade observations to focus on more reliable data.

This function is essential for preparing the bond price data for accurate calculation of financial metrics such as illiquidity.

In [8]:
# Calculate price change data using periods in the paper
df_paper = calc_illiquidity.calc_deltaprc(cleaned_df_paper)
df_paper.head()
Out[8]:
cusip trd_exctn_dt prclean month_time date price_eom tmt t_volume t_dvolume t_spread ... offering_date year month_year trd_exctn_dt_lag n logprc logprc_lag deltap logprc_lead deltap_lag
1225 345397SQ7 2003-04-16 103.468001 2003-04 2003-04-30 105.029447 1.902778 532280000.0 5.516352e+08 0.003911 ... 2000-03-09 2003 2003-04 2003-04-15 1.0 4.639262 4.633919 0.534345 4.639275 0.001255
1226 370425RJ1 2003-04-16 104.210101 2003-04 2003-04-30 104.532582 1.150000 132358000.0 1.383409e+08 0.002930 ... 1999-06-10 2003 2003-04 2003-04-15 1.0 4.646409 4.646475 -0.006620 4.647230 0.082107
1227 260543BU6 2003-04-16 103.132500 2003-04 2003-04-30 103.512000 4.611111 68490000.0 7.049571e+07 0.005718 ... 2002-11-18 2003 2003-04 2003-04-15 1.0 4.636015 4.627253 0.876200 4.629933 -0.608155
1228 191219BF0 2003-04-16 111.127899 2003-04 2003-04-30 111.101975 5.588889 14511000.0 1.605824e+07 0.008366 ... 1998-10-28 2003 2003-04 2003-04-15 1.0 4.710682 4.692768 1.791380 4.695206 -1.547620
1229 37042GD52 2003-04-16 96.987273 2003-04 NaT NaN NaN NaN NaN NaN ... NaN 2003 2003-04 2003-04-15 1.0 4.574580 4.585009 -1.042888 4.574582 0.000207

5 rows × 29 columns

In [9]:
# Calculate price change data using update-to-date periods
df_new = calc_illiquidity.calc_deltaprc(cleaned_df_new)
df_new.head()
Out[9]:
cusip trd_exctn_dt prclean month_time date price_eom tmt t_volume t_dvolume t_spread ... offering_date year month_year trd_exctn_dt_lag n logprc logprc_lag deltap logprc_lead deltap_lag
1185 079857AG3 2003-04-16 115.000004 2003-04 2003-04-30 120.794858 6.897222 65637000.0 7.903382e+07 0.010070 ... 2000-02-11 2003 2003-04 2003-04-15 1.0 4.744932 4.785665 -4.073293 4.782680 3.774806
1186 00184AAD7 2003-04-16 104.440100 2003-04 2003-04-30 105.078814 2.033333 188899000.0 1.983920e+08 0.005451 ... 2002-04-03 2003 2003-04 2003-04-15 1.0 4.648614 4.647722 0.089183 4.650268 0.165414
1187 38141GBU7 2003-04-16 111.903098 2003-04 2003-04-30 112.786977 8.838889 460039000.0 5.139233e+08 0.005054 ... 2002-01-03 2003 2003-04 2003-04-15 1.0 4.717633 4.713332 0.430137 4.721193 0.355922
1188 36962GZH0 2003-04-16 104.363098 2003-04 2003-04-30 104.891028 6.472222 134213500.0 1.402319e+08 0.004858 ... 2002-09-18 2003 2003-04 2003-04-15 1.0 4.647876 4.649814 -0.193753 4.650259 0.238308
1189 025816AK5 2003-04-16 105.940000 2003-04 2003-04-30 105.920706 1.166667 17519000.0 1.856660e+07 0.003665 ... 1997-06-18 2003 2003-04 2003-04-15 1.0 4.662873 4.658340 0.453262 4.661254 -0.161921

5 rows × 29 columns

Step 3: Panel A Individual Bond: Illiquidity Metrics Calculation Using Daily Bond Data¶

This step involves using the calc_annual_illiquidity_table_ function to calculate and summarize annual illiquidity metrics for corporate bonds. The function takes daily bond data as input and computes several statistics that capture the illiquidity of bonds on an annual basis. create_annual_illiquidity_table function is used as the last step in calc_annual_illiquidity_table to generate illiquidity table with significance percentage, robust t-stat, mean and median.

  • Computes the illiquidity for each bond by month by taking the negative of the covariance between daily price changes (deltap) and their lagged values (deltap_lag).

  • Aggregated the monthly illiquidity measures to obtain annual statistics, including mean and median illiquidity.

  • Calculates t-statistics for the mean illiquidity of each bond and year and determines the percentage of these t-stats that are significant (>= 1.96).

  • Calculates robust t-stats are calculated using OLS with HAC (heteroskedasticity and autocorrelation consistent) standard errors.

  • Calculate overall statistics across the full sample period.

  • Compiles all these metrics into a table that presents the mean and median illiquidity, the percentage of significant t-statistics, and robust t-statistics for each year, as well as for the full sample period.

This comprehensive illiquidity metric calculation allows us to understand the annual and overall liquidity characteristics of the corporate bond market.

Replication¶

Table 2 Panel A Daily Data¶

During the period in the paper spanning from 2003 to 2009, the illiquidity metric γ exhibited a mean value of 3.12 and a median of 0.07, with a substantial t-statistic of 17.06 using daily data, compared to an average of 1.18 and a median of 0.56 observed in the paper. Our analysis successfully mirrored the initial decline followed by a subsequent rise in trends as documented in the original study. While other illiquidity metrics maintained a deviation within 40% when compared to the original findings, the illiquidity we recorded for 2008-2009 were significantly higher—by a factor of 3 to 4 times—potentially influenced by approximately six bonds exhibiting γ values exceeding 2000. The original study, however, did not specify an approach for managing outliers, leaving us uncertain whether these variations arise from outlier effects or inherent differences in data. In addition, our percentage of illiquidity significant at 95% level is much lower than what the paper has, suggesting that the authors might have handled outliers somewhat differently to maintain higher significance. 6 out of 8 robust t-stats are significant at 95% level in our analysis, with the overall robust t-stat = 17.6, close to the 16.53 in the paper, indicating the overall significance of the data.

In [10]:
# Replicate table 2 panel A daily data in the paper
illiq_daily_paper, table2_daily_paper = calc_illiquidity.calc_annual_illiquidity_table(df_paper)
table2_daily_paper

100%|██████████| 47630/47630 [00:07<00:00, 6107.46it/s]
Out[10]:
Year Mean illiq Median illiq Per t greater 1.96 Robust t stat
0 2003 1.012431 0.118649 77.283913 2.541571
1 2004 1.054907 0.061028 77.377015 10.183842
2 2005 0.851126 0.040949 80.297199 4.067339
3 2006 0.409025 0.036393 87.914056 6.715445
4 2007 1.116023 0.064962 87.326335 1.815385
5 2008 13.271568 0.232770 67.408497 20.787846
6 2009 17.980450 0.334487 69.007369 1.186281
7 Full 3.119932 0.072591 79.679268 17.057181
In [11]:
# Update table 2 panel A daily data to the present
illiq_daily_new, table2_daily_new = calc_illiquidity.calc_annual_illiquidity_table(df_new)
table2_daily_new

100%|██████████| 62473/62473 [00:10<00:00, 5714.14it/s]
Out[11]:
Year Mean illiq Median illiq Per t greater 1.96 Robust t stat
0 2003 1.013071 0.117607 77.850215 0.490985
1 2004 1.070753 0.060145 77.723404 6.150653
2 2005 0.843624 0.039453 80.766689 2.635933
3 2006 0.417981 0.034866 87.788779 0.984113
4 2007 1.128247 0.062519 87.945643 6.522312
5 2008 10.519394 0.219249 67.834456 0.374275
6 2009 4.010328 0.182663 73.894334 15.856989
7 2010 0.444072 0.051685 91.007616 40.595994
8 2011 0.379700 0.036848 82.629630 28.768055
9 2012 0.267179 0.041479 92.354884 0.140632
10 2013 1.360981 0.031011 89.483997 3.170728
11 2014 0.222589 0.059653 91.860465 0.604964
12 2015 0.408777 0.193111 94.977169 10.270055
13 2016 0.558835 0.231420 95.365419 1.495317
14 2017 0.206552 0.144149 97.042514 17.320413
15 2018 2.683180 0.115723 88.910506 2.622831
16 2019 0.252325 0.139259 86.492891 7.210641
17 2020 1.191521 0.231415 26.801153 2.925778
18 2021 0.185068 0.081809 73.846154 1.512545
19 2022 0.487813 0.157808 73.765432 0.936680
20 Full 1.821362 0.069791 81.853406 176.451616

Step 4: Summary Statistics Compilation Using Daily Illiquidity Data¶

This step entails utilizing the create_summary_stats function to compile key summary statistics that characterize daily illiquidity data for corporate bonds over different years--min, mean, median, max, 25%, 75% std monthly illiquidity per cusip and mean t-stat. This aids in understanding the distribution and central tendencies of bond illiquidity and t-statistics on an annual basis.

In [12]:
# Produce summary stats for per bond monthly illiquidity using periods in the paper
illiq_daily_summary_paper = calc_illiquidity.create_summary_stats(illiq_daily_paper)
illiq_daily_summary_paper
Out[12]:
year min illiq mean illiq q1 0.25 median q3 0.75 max illiq std illiq mean t stat
0 2003 -129.009527 1.012431 0.030657 0.118649 0.416826 1127.480854 15.926022 2.925198
1 2004 -6.984261 1.054907 0.014045 0.061028 0.242994 718.689577 17.546287 3.040041
2 2005 -12.645612 0.851126 0.009378 0.040949 0.171790 2116.809197 29.728685 3.109209
3 2006 -20.151911 0.409025 0.007929 0.036393 0.161738 787.236482 9.327834 3.437520
4 2007 -9.345427 1.116023 0.015308 0.064962 0.240230 1764.116960 26.605177 3.248946
5 2008 -830.664798 13.271568 0.059508 0.232770 1.132950 5836.747235 190.986134 2.517271
6 2009 -202.045271 17.980450 0.070607 0.334487 2.013726 8571.428571 233.059386 2.654317
In [13]:
# Examine outliers
illiq_daily_paper[illiq_daily_paper['illiq'] > 2000]
Out[13]:
cusip month_year illiq year t stat significant
15350 247126AE5 2009-04 2780.997196 2009 1.176218 False
15370 247361YE2 2005-10 2116.809197 2005 1.020795 False
17130 26632QAH6 2008-01 2001.066833 2008 3.287915 True
17132 26632QAH6 2008-03 5434.541339 2008 3.287915 True
17133 26632QAH6 2008-04 5836.747235 2008 3.287915 True
17134 26632QAH6 2008-05 5758.264060 2008 3.287915 True
17138 26632QAH6 2008-09 3333.333371 2008 3.287915 True
17140 26632QAH6 2008-12 5714.285714 2008 3.287915 True
17160 26632QAK9 2008-06 2037.730503 2008 0.797720 False
27609 432848AS8 2008-12 3560.187974 2008 1.172602 False
46095 939322AN3 2008-10 2921.702133 2008 1.195380 False
46225 93933WAA4 2008-11 3810.208059 2008 1.404210 False
46226 93933WAA4 2008-12 2128.962866 2008 1.404210 False
46228 93933WAA4 2009-02 2500.000005 2009 3.208985 True
46229 93933WAA4 2009-03 4164.020769 2009 3.208985 True
46230 93933WAA4 2009-04 3497.572243 2009 3.208985 True
46231 93933WAA4 2009-05 2047.757553 2009 3.208985 True
46232 93933WAA4 2009-06 8571.428571 2009 3.208985 True
In [14]:
# Produce summary stats for per bond monthly illiquidity using update-to-date periods
illiq_daily_summary_new = calc_illiquidity.create_summary_stats(illiq_daily_new)
illiq_daily_summary_new
Out[14]:
year min illiq mean illiq q1 0.25 median q3 0.75 max illiq std illiq mean t stat
0 2003 -129.009527 1.013071 0.030905 0.117607 0.407148 1127.480854 16.139634 2.929592
1 2004 -6.984261 1.070753 0.014037 0.060145 0.241248 718.689577 17.767299 3.051168
2 2005 -12.645612 0.843624 0.009305 0.039453 0.167497 2116.809197 30.102278 3.139655
3 2006 -20.151911 0.417981 0.007769 0.034866 0.155547 787.236482 9.478433 3.451152
4 2007 -8.932729 1.128247 0.014896 0.062519 0.226398 1764.116960 27.067324 3.264730
5 2008 -284.127247 10.519394 0.058612 0.219249 1.013677 5836.747235 172.631333 2.562620
6 2009 -162.993465 4.010328 0.041266 0.182663 1.104113 883.596573 23.289963 2.687941
7 2010 -32.626281 0.444072 0.015076 0.051685 0.242857 54.345286 2.248481 3.739615
8 2011 -2.394000 0.379700 0.010355 0.036848 0.216858 17.530678 1.235823 3.355690
9 2012 -20.161320 0.267179 0.009667 0.041479 0.219475 36.981744 1.474108 3.783989
10 2013 -47.870385 1.360981 0.004976 0.031011 0.171623 1605.895174 41.582784 3.563826
11 2014 -0.674958 0.222589 0.006265 0.059653 0.264690 11.441135 0.513601 4.014405
12 2015 -5.307065 0.408777 0.042627 0.193111 0.552432 5.349022 0.664056 4.193873
13 2016 -0.592674 0.558835 0.065495 0.231420 0.682004 9.292445 0.882031 3.961721
14 2017 -52.276911 0.206552 0.033942 0.144149 0.306017 12.975677 2.398462 3.883913
15 2018 -26.542294 2.683180 0.024995 0.115723 0.324668 813.959483 37.751823 3.573085
16 2019 -2.822076 0.252325 0.041604 0.139259 0.306641 6.501283 0.535831 3.401772
17 2020 -9.728384 1.191521 0.047967 0.231415 0.840038 67.357836 4.885321 1.336427
18 2021 -1.244749 0.185068 0.015033 0.081809 0.229897 4.019345 0.371454 3.188224
19 2022 -3.855912 0.487813 0.029016 0.157808 0.475796 10.091643 1.163633 2.867860

Step 5: Panel A Using MMN Corrected Daily Bond Data¶

Now, we apply similar calculation in Step 3 and 4 using MMN corrected daily bond data. Since the MMN corrected daily bond data contains illiquidty directly, calc_illiq_w_mmn_corrected performs cleaning on MMN corrected data and apply create_annual_illiquidity_table to generate the similar Panel A (daily data) illiquidity final table, ready for comparison. We then use the in Step 4 to produce summary stats using cleaned MMN corrected daily bond data.

In [15]:
# Replicate table 2 panel A daily data in the paper using MMN corrected data
mmn_paper, table2_daily_mmn_paper = calc_illiquidity.calc_illiq_w_mmn_corrected(
    start_date, end_date, cleaned_df_paper)
table2_daily_mmn_paper
Out[15]:
Year Mean illiq Median illiq Per t greater 1.96 Robust t stat
0 2003 1.126366 0.107766 71.480204 2.304529
1 2004 1.351033 0.061672 71.377686 10.132517
2 2005 0.433418 0.045342 79.905201 5.575357
3 2006 0.329079 0.045745 83.898432 0.497943
4 2007 0.403477 0.080262 88.265670 0.774973
5 2008 4.387134 0.269606 61.268378 10.143288
6 2009 8.436064 0.424655 63.152455 0.823691
7 Full 1.507753 0.078402 75.995184 7.206051
In [16]:
# Produce summary stats for per bond monthly illiquidity using periods in the paper using MMN corrected data
illiq_daily_summary_mmn_paper = calc_illiquidity.create_summary_stats(mmn_paper)
illiq_daily_summary_mmn_paper
Out[16]:
year min illiq mean illiq q1 0.25 median q3 0.75 max illiq std illiq mean t stat
0 2003 -9.143287 1.126366 0.025527 0.107766 0.345540 751.217416 16.797166 2.733560
1 2004 -20.038684 1.351033 0.013673 0.061672 0.247137 778.589263 21.198697 2.832352
2 2005 -18.137484 0.433418 0.010328 0.045342 0.178552 926.156664 12.341459 2.995291
3 2006 -40.331088 0.329079 0.010483 0.045745 0.167151 550.736267 7.142692 3.299684
4 2007 -4.427193 0.403477 0.022036 0.080262 0.272474 500.221470 6.729276 3.223165
5 2008 -249.784908 4.387134 0.065605 0.269606 1.129595 1084.715175 34.324809 2.246503
6 2009 -73.678064 8.436064 0.086623 0.424655 2.289379 925.409177 46.130775 2.377221
In [17]:
mmn_paper.head()
Out[17]:
date cusip exretn_t+1 exretnc_dur_t+1 bond_ret_t+1 bond_ret exretn exretnc_dur rating cs ... BONDPRC PRFULL DURATION CONVEXITY bond_value BOND_VALUE year illiq t stat significant
27967 2003-04-30 001546AE0 -0.063575 -0.082483 -0.062675 -0.023885 -0.024885 -0.025599 12.0 0.078802 ... 92.6618 94.258673 4.427331 24.702911 40122632.0 41697810.0 2003 0.647163 2.245310 True
27978 2003-04-30 00184AAA3 0.017444 0.009376 0.018344 0.043164 0.042164 0.041512 8.0 0.016238 ... 107.1509 107.423125 2.706638 8.983116 107524651.0 107150900.0 2003 0.175779 2.033155 True
27979 2003-04-30 00184AAB1 0.050186 0.020915 0.051086 0.025243 0.024243 0.023114 8.0 0.023513 ... 107.8086 108.108602 6.168393 46.752979 108353755.0 107808600.0 2003 0.071140 1.487224 False
27980 2003-04-30 00184AAC9 0.082945 0.037654 0.083845 0.021796 0.020796 0.019919 8.0 0.029814 ... 109.0437 109.382588 12.075853 231.670907 216107272.0 218087400.0 2003 -0.163314 1.816756 False
27981 2003-04-30 00184AAD7 0.010570 0.007493 0.011470 0.010952 0.009952 0.009416 8.0 0.015651 ... 103.5500 103.549999 1.888003 4.574499 105078814.0 103550000.0 2003 0.069521 3.025802 True

5 rows × 37 columns

In [18]:
# Update table 2 panel A daily data to the present using MMN corrected data
mmn_new, table2_daily_mmn_new = calc_illiquidity.calc_illiq_w_mmn_corrected(
    start_date, today, cleaned_df_new)
table2_daily_mmn_new
Out[18]:
Year Mean illiq Median illiq Per t greater 1.96 Robust t stat
0 2003 1.263910 0.110032 72.102718 8.033270
1 2004 1.495441 0.064419 71.634726 2.312737
2 2005 0.465774 0.046236 79.585983 4.800168
3 2006 0.342268 0.046322 83.994630 1.502008
4 2007 0.414206 0.082518 89.803157 9.309214
5 2008 4.418317 0.269047 60.336279 48.680247
6 2009 4.661818 0.227257 70.217853 50.570631
7 2010 0.247482 0.066234 88.923077 6.462690
8 2011 0.288858 0.047112 79.589041 12.204280
9 2012 0.280102 0.066420 90.576271 1.246858
10 2013 0.196855 0.060676 82.881002 0.938458
11 2014 0.254639 0.111558 84.640000 1.168828
12 2015 0.400453 0.204659 84.240150 8.534110
13 2016 0.469609 0.160900 80.722892 2.672773
14 2017 0.192422 0.086299 80.875576 6.721826
15 2018 0.188212 0.071414 70.165746 13.375011
16 2019 0.144252 0.079675 77.945619 2.958322
17 2020 0.699928 0.076966 12.121212 0.379296
18 2021 0.077082 0.027756 66.315789 3.522953
19 2022 0.175075 0.060517 54.545455 3.297673
20 Full 1.280751 0.079629 77.356168 8.695623
In [19]:
# Produce summary stats for per bond monthly illiquidity using update-to-date periods using MMN corrected data
illiq_daily_summary_mmn_new = calc_illiquidity.create_summary_stats(mmn_new)
illiq_daily_summary_mmn_new
Out[19]:
year min illiq mean illiq q1 0.25 median q3 0.75 max illiq std illiq mean t stat
0 2003 -9.357964 1.263910 0.026446 0.110032 0.361274 767.255311 19.564480 2.756077
1 2004 -20.038684 1.495441 0.014128 0.064419 0.260289 778.589263 23.087278 2.846538
2 2005 -18.137484 0.465774 0.010503 0.046236 0.185411 926.156664 12.357977 3.018792
3 2006 -40.331088 0.342268 0.010857 0.046322 0.172629 550.736267 7.142619 3.312651
4 2007 -1.784522 0.414206 0.022977 0.082518 0.280699 500.221470 6.718872 3.258224
5 2008 -249.784908 4.418317 0.065406 0.269047 1.099317 1084.715175 34.908861 2.268795
6 2009 -73.678064 4.661818 0.051912 0.227257 1.125256 925.409177 32.378854 2.473155
7 2010 -6.315249 0.247482 0.019890 0.066234 0.217277 19.621526 0.710688 3.439110
8 2011 -11.126745 0.288858 0.012429 0.047112 0.226043 19.924739 1.073702 3.175623
9 2012 -1.646094 0.280102 0.016928 0.066420 0.270844 26.350945 0.867468 3.707704
10 2013 -3.283444 0.196855 0.011228 0.060676 0.231889 7.837551 0.436248 3.283463
11 2014 -0.887116 0.254639 0.030336 0.111558 0.313058 3.500040 0.407492 3.889046
12 2015 -8.736595 0.400453 0.052067 0.204659 0.554114 6.739926 0.831134 3.520315
13 2016 -4.284312 0.469609 0.044744 0.160900 0.510266 11.642210 1.050132 3.107543
14 2017 -0.416079 0.192422 0.021377 0.086299 0.232659 4.298830 0.405718 3.193291
15 2018 -2.673262 0.188212 0.015354 0.071414 0.188523 19.944420 1.090180 2.912958
16 2019 -0.497105 0.144252 0.013489 0.079675 0.186222 3.347248 0.304816 2.889233
17 2020 -9.254585 0.699928 0.006878 0.076966 0.297700 34.116047 3.334329 0.707914
18 2021 -0.143080 0.077082 0.000572 0.027756 0.087069 1.846224 0.178311 2.147115
19 2022 -2.066814 0.175075 -0.013492 0.060517 0.199545 5.457241 0.583662 1.350330

Step 6: Panel B Bond Portfolios: Portfolio-Based Annual Illiquidity Metrics Calculation¶

The calc_annual_illiquidity_table_portfolio function computes the illiquidity metrics for corporate bonds by constructing equal-weighted and issuance-weighted portfolio returns on a daily basis and then calculate portfolio illiquidity on an annual basis. The function systematically processes transaction-level bond data to assess market liquidity through portfolio aggregation, offering a more holistic view of the market dynamics.

  • Equal-Weighted Portfolio Calculation: Creat an equal-weighted portfolio for each trading day by averaging the daily price changes (deltap) and their lagged values (deltap_lag). It then groups these daily averages by year to calculate the negative covariance between the deltap and deltap_lag to derive the illiquidity measure for each year. Additionally, a t-statistic for the mean illiquidity of the equal-weighted portfolio is computed.

  • Issuance-Weighted Portfolio Calculation: Each bond is calculated with its $ \text{issuance} = \text{offering amount} \times \text{principal amount} \times \text{offering price} / 100 / 1,000,000 $ , and all bonds deltap and deltap_lag are aggregated on a daily basis weighted by issurance. The following steps are similar to Equal-Weighted Portfolio Calculation.

  • Calculate overall statistics across the full sample period.

  • Compiles all these metrics into a table that presents the mean equal_weighted portfolio and t-stat, mean issuance-weighted portfolio illiquidity and t-stat for each year, as well as for the full sample period.

Replication¶

Table 2 Panel B Bond Portfolio¶

For Panel B, we are trying to construct two sets of daily bond bond portfolios from the same cross-section of bonds and for the same sample period, one being equally weighted and the other being weighted by issuance. After obtaining the daily portfolio returns (using delta log bond price) and lag returns (using delta log bond price lag), we calculated the monthly illiquidity through negative covariance of the returns and lag returns and then found the median per year for two sets of portfolios.

The paper suggests that this measure implies that the transitory component extracted by the γ measure is idiosyncratic in nature and gets diversified away at the portfolio level, but a suspected systematic component is present when this aggregate illiquidity measure comoves strongly with the aggregate market condition at the time. Similar to the paper, our peak in illiquidity appeared in ~2006-2007, and most of the portfolio illiquidity measures were not statistically significant. All measures replicate the paper within a tolerance of +-0.05 (equal-weighted), +-0.07(issuance-weighted).

In [20]:
# Replicate table 2 panel B in the paper
table2_port_paper = calc_illiquidity.calc_annual_illiquidity_table_portfolio(df_paper)
table2_port_paper

100%|██████████| 75/75 [00:00<00:00, 5768.70it/s]
100%|██████████| 75/75 [00:00<00:00, 5400.48it/s]
Out[20]:
Year Equal weighted EW t stat Issuance weighted IW t stat
0 2003 0.006096 1.454041 0.006475 0.040642
1 2004 -0.000824 -0.541979 -0.000897 -0.736524
2 2005 0.000017 -0.256471 -0.000779 -0.890187
3 2006 0.000968 -0.195230 0.000129 0.193256
4 2007 0.000577 1.555363 0.001253 0.475629
5 2008 -0.000293 -0.095826 0.001240 -1.178410
6 2009 -0.008024 -0.954207 -0.017548 -2.180456
7 Full 0.000757 -0.294414 0.000235 -1.706940
In [21]:
# Update table 2 panel B to the present
table2_port_new = calc_illiquidity.calc_annual_illiquidity_table_portfolio(df_new)
table2_port_new

100%|██████████| 237/237 [00:00<00:00, 6489.34it/s]
100%|██████████| 237/237 [00:00<00:00, 5764.41it/s]
Out[21]:
Year Equal weighted EW t stat Issuance weighted IW t stat
0 2003 0.005872 1.513927 0.007165 0.083672
1 2004 0.001517 -0.341734 -0.000989 -0.727405
2 2005 -0.001277 -0.705159 -0.001579 -0.538040
3 2006 0.000955 -0.161202 0.000185 0.119966
4 2007 0.001051 1.615810 0.001458 0.361957
5 2008 -0.003147 -0.319423 -0.000732 -1.023535
6 2009 -0.003195 -1.619786 -0.007418 -1.633515
7 2010 -0.000656 -1.611385 0.001214 0.182648
8 2011 0.001299 0.597844 0.001089 -0.135057
9 2012 0.002561 2.319715 0.002139 1.197730
10 2013 0.001261 1.545646 0.001295 0.701735
11 2014 0.003317 1.679030 0.004680 2.098618
12 2015 0.021072 2.688333 0.025910 3.131265
13 2016 0.008485 1.280233 0.019069 2.160368
14 2017 0.004978 1.716060 0.020470 2.704882
15 2018 0.030206 2.783250 0.008720 0.928023
16 2019 0.010155 2.476211 0.021513 2.668051
17 2020 0.000651 -1.189270 0.014805 -0.909053
18 2021 0.005823 1.640610 0.018096 1.523730
19 2022 0.018860 1.419173 0.022416 1.086828
20 Full 0.002231 -0.516261 0.002236 -0.464044

Step 7: Panel C Implied by Quoted Bid-Ask Spreads: Annual Implied Illiquidity Using Monthly Quoted Bid-Ask Spread¶

In this section, we focus on analyzing the illiquidity implied by quoted bid-ask spreads of corporate bonds on an annual basis using calc_annual_illiquidity_table_spd.

  • For each year, calculates the mean and median of the monthly t_spread, which represent the implied gamma.

  • Calculate overall statistics across the full sample period.

  • Compiles all these metrics into a table that presents the mean and median implied illiquidity for each year, as well as for the full sample period.

By computing these statistics, the function provides insights into the liquidity of the corporate bond market as implied by the bid-ask spreads over time. As shown in the paper, not only does the quoted bid-ask spread fail to capture the overall level of illiquidity, but it also fails to explain the cross-sectional variation in bond illiquidity and its asset pricing implications.

Replication¶

Table 2 Panel C Bid-Ask Spread¶

In Panel C, we computed the monthly average and median bid-ask spreads for each year, using these as proxies for implied illiquidity. The methodology involved utilizing the monthly bond return data available on WRDS to calculate the t-spreads, whereas the original authors derived their data from daily figures, potentially accounting for some differences in results. Despite these differences, by applying a factor of 5 to our findings, we were able to align our results with the original study's observed pattern of initial decline followed by an increase in illiquidity, with a tolerance level below 40%. It is noteworthy that the mean bid-ask spread for 2005 exhibited a slight increase in our table, although the median remained lower than that of the preceding year. This discrepancy underscores the influence of outliers on the mean and indicates a positive skew in the data.

In [22]:
# Replicate table 2 panel C in the paper
table2_spd_paper = calc_illiquidity.calc_annual_illiquidity_table_spd(df_paper) 
table2_spd_paper
Out[22]:
Year Mean implied gamma Median implied gamma
0 2003 0.006595 0.004536
1 2004 0.005371 0.003603
2 2005 0.004684 0.003305
3 2006 0.004274 0.003142
4 2007 0.005705 0.004373
5 2008 0.012404 0.008957
6 2009 0.015906 0.012258
7 Full 0.006560 0.004239
In [23]:
# Update table 2 panel C to the present
table2_spd_new = calc_illiquidity.calc_annual_illiquidity_table_spd(df_new) 
table2_spd_new
Out[23]:
Year Mean implied gamma Median implied gamma
0 2003 0.006659 0.004563
1 2004 0.005447 0.003603
2 2005 0.004749 0.003290
3 2006 0.004380 0.003110
4 2007 0.005767 0.004323
5 2008 0.012336 0.008892
6 2009 0.012398 0.009269
7 2010 0.006056 0.004523
8 2011 0.005425 0.003472
9 2012 0.005628 0.003654
10 2013 0.005210 0.003580
11 2014 0.006207 0.004951
12 2015 0.008238 0.007859
13 2016 0.009391 0.008588
14 2017 0.007152 0.006530
15 2018 0.006422 0.005887
16 2019 0.006709 0.006065
17 2020 0.007229 0.005632
18 2021 0.004489 0.004014
19 2022 0.005939 0.004894
20 Full 0.006547 0.004355

Step 8: Plot Monthly Illiquidity Per Bond and Average Illiquidity By Year¶

The plot_illiquidity function visualizes both monthly bond illquidity observations and annual trends.

  • Monthly Illiquidity Per Bond: This granular data paves the way for an in-depth examination of liquidity at the bond level, month by month, in scatter.

  • Annual Illiquidity Summary Insights: Visualize Table 2 Panel 1 mean and median illiquidity using daily data, shown as the lines on the plot. Red line indicates the mean, purple line the median. Mean is much higher than median around ~2008-2009, suggesting high illquidity outliers.

  • The Zoomed-In Analysis: Acknowledging the potential distortion by extreme values, the function prudently narrows down the focus in the second subplot. By honing in on a more typical range of illiquidity values, it effectively filters out the outliers, thereby furnishing a clearer, more focused analysis of the prevalent liquidity patterns.

We have used both original data and MMN corrected data to generate seperate plots.

In [24]:
def plot_illiquidity_plotly(illiquidity_df, summary_df, title):
    """Plot monthly illiquidity per bond and average & median illiquidity by year,
    using plotly for interactive features.
    
    Parameters:
        illiquidity_df (pandas.DataFrame): Monthly illiquidity per bond dataframe.
        summary_df (pandas.DataFrame): Summary stats for illiquidity per year. 
        title (str): Desired plot title.

    Returns:
        Interactive plotly plot to visualize illiquidity.
    
    """

    fig = make_subplots(rows=1, cols=1, subplot_titles=(f'Illiquidity by Year with Mean Illiquidity, {title}'))

    if 'date' in list(illiquidity_df.columns):
        illiquidity_df['month_year'] = illiquidity_df['date']

    fig = px.scatter(illiquidity_df, x="month_year", y="illiq")

    fig.add_trace(go.Scatter(x=summary_df['year'], y=summary_df['mean illiq'],
                                mode='lines', name='Mean Illiquidity', line=dict(color='red')),
                    row=1, col=1)

    fig.add_trace(go.Scatter(x=summary_df['year'], y=summary_df['median'],
                                mode='lines', name='Median Illiquidity', line=dict(color='purple')),
                    row=1, col=1)
        
    # Update x/y-axis properties
    fig.update_xaxes(title_text='Year', row=1, col=1)
    fig.update_yaxes(title_text='Illiquidity', row=1, col=1)
    fig.update_layout(height=500, showlegend=True, title_text=f"Illiquidity Analysis: {title}")
    fig.show()
In [25]:
# Plot using original data, 2003-2009
illiq_daily_paper = pd.read_csv(OUTPUT_DIR / "illiq_daily_paper.csv")
illiq_daily_summary_paper = pd.read_csv(OUTPUT_DIR / "illiq_summary_paper.csv")

plot.plot_illiquidity(illiq_daily_paper, illiq_daily_summary_paper, "2003-2009")
No description has been provided for this image
In [26]:
plot_illiquidity_plotly(illiq_daily_paper, illiq_daily_summary_paper, '2003-2009')
In [27]:
# Plot using original data, 2003-2023
illiq_daily_new = pd.read_csv(OUTPUT_DIR / "illiq_daily_new.csv")
illiq_daily_summary_new = pd.read_csv(OUTPUT_DIR / "illiq_summary_new.csv")
    
plot.plot_illiquidity(illiq_daily_new, illiq_daily_summary_new, "2003-2023")
No description has been provided for this image
In [28]:
plot_illiquidity_plotly(illiq_daily_new, illiq_daily_summary_new, '2003-2023')
In [29]:
# Plot using MMN corrected data, 2003-2009
mmn_paper = pd.read_csv(OUTPUT_DIR / "mmn_paper.csv")
illiq_daily_summary_mmn_paper = pd.read_csv(OUTPUT_DIR / "illiq_daily_summary_mmn_paper.csv")

plot.plot_illiquidity(mmn_paper, illiq_daily_summary_mmn_paper, "MMN_Corrected, 2003-2009")
No description has been provided for this image
In [30]:
plot_illiquidity_plotly(mmn_paper, illiq_daily_summary_mmn_paper, 'MMN Corrected Data, 2003-2009')
In [31]:
# Plot using MMN corrected data, 2003-2023
mmn_new = pd.read_csv(OUTPUT_DIR / "mmn_new.csv")
illiq_daily_summary_mmn_new = pd.read_csv(OUTPUT_DIR / "illiq_daily_summary_mmn_new.csv")

plot.plot_illiquidity(mmn_new, illiq_daily_summary_mmn_new, "MMN_Corrected, 2003-2023")
No description has been provided for this image
In [32]:
plot_illiquidity_plotly(mmn_new, illiq_daily_summary_mmn_new, 'MMN Corrected Data, 2003-2023')